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Incremental  development  entails  the  deliberate  deferral  of 
work  to  a  subsequent  period,  using  technology  maturity  as  the 
measure  of  readiness.  This  article  illustrates  that  this  approach 
might  enable  more  effective  delivery  of  the  first  increment 
with  a  comparison  of  two  major  systems  as  case  studies.  But 
there  are  some  inherent  risks  in  an  evolutionary  approach 
as  well,  and  the  authors  caution  that  certain  attributes  of 
hardware  products  might  help  determine  the  suitability  of 
evolutionary  development  methodologies.  Mutable  prod¬ 
ucts  with  costless  production,  continuous  requirements,  low 
maintenance,  or  time  criticality  may  be  more  likely  to  reap 
advantages  from  evolutionary  approaches.  Products  that  are 
nearly  immutable,  have  binary  requirements  for  key  capabili¬ 
ties,  require  man-rating,  or  are  maintenance-intensive  may 
not  be  best  candidates  for  incremental  development. 
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Since  his  work  in  the  1830s,  Charles  Darwin  has  received  much  of  the 
credit  for  furthering  a  theory  of  biological  evolution.  While  not  the  first  to 
have  the  idea,  he  associated  observations  of  species  variety  on  the  island  of 
Galapagos  with  species  environment,  and  suggested  that  nature  selected  the 
variations  that  were  the  fittest  (Darwin,  1859).  In  its  time  (and  since),  the  idea 
was  considered  radical  and  a  threat  to  religious  and  social  order.  Mere  variety 
itself  can  be  controversial  since,  paradoxically,  variety  is  appreciated  in  some 
domains  (Ashby,  1960)  and  abhorred  in  others  (Neave,  2000). 

At  the  center  of  evolutionary  acquisition  are  also  ideas  and  phenomena 
about  variety  and  change.  As  a  policy  for  system  development,  it  too  is 
controversial.  And  as  within  Darwinian  concepts,  product  evolution  involves 
information  transfer,  interaction  with  the  environment,  and  unpredictability 
of  change  outcomes.  But  unlike  evolutionary  biology,  product  variations  and 
selections  occur  frequently  and  are  non-random.  Much  of  what  we  have  found  in 
the  following  research  on  evolutionary  development  and  project  management 
is  about  how  managers  must  cope  with  product  variety  and  change.  Using 
case  study  analyses,  review  of  current  subject  literature,  and  computational 
modeling  (expounded  upon  in  a  companion  article:  Ford  &  Dillard,  2009),  the 
focus  of  our  research  was  to  ascertain  the  program  management  implications  of 
evolutionary  acquisition,  obtain  lessons  learned  in  past  programs  as  applicable 
to  future  development  efforts,  model  and  simulate  projects  that  used  different 
acquisition  approaches,  derive  predictions,  and  make  recommendations  to 
project  managers  for  the  effective  and  efficient  harnessing  and  implementation 
of  evolutionary  acquisition. 


Background 

The  Department  of  Defense  (DoD)  promulgated  evolutionary  acquisition 
(EA)  as  policy  in  2000,  and  soon  after,  spiral  development  for  the  preferred 
acquisition  strategy  of  all  materiel  (Office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics,  2000).  EA’s  goal  is  to  time-phase  system 
requirements  and  provide  capabilities  sooner.  But  confusion  over  terms  persists, 
despite  further  elaboration  and  even  codification  in  statute  (Armed  Forces,  2002), 
along  with  a  lack  of  full  understanding  of  many  policy  implications— especially 
some  inherent  risks.  DoD  policy  for  evolutionary  acquisition  mandated  spiral 
(i.e.,  amorphous  and  unplanned  requirements/technologies)  or  incremental  (i.e., 
defined  and  deferred  requirements/technologies)  development  methodologies 
for  all  programs.  Since  all  amorphous  spirals  eventually  become  defined 
increments,  the  disappearance  of  this  term  “spiral”  in  most  recent  (2008)  policy 
will  not  be  missed. 

Fundamentally,  E  A  means  there  will  always  be  multiple  product  releases  of  an 
item.  The  current  policy  thrust  is  primarily  about  the  reduction  of  product  cycle 
time  within  an  uncertain  environment  by  using  mature  technology  exclusively.  The 
DoD’s  requirements  process  has  also  followed  with  “evolutionary”  requirements 
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documents— a  new  idea.  Uncertainty  is  the  usual  realm  of  program  managers 
(PM),  especially  in  defense  systems,  and  is  usually  dealt  with  by  seeking  best 
information  (Pich,  Loch,  &  De  Meyer,  2002).  Earlier  reform  initiatives  were  aimed 
at  overcoming  information  gaps  and  technology  lag.  For  example,  the  1990s 
Integrated  Product  and  Process  Development  initiative  was  about  collaboration 
for  early  and  complete  requirements  realization.  However,  the  current  paradigm 
is  to  allow  uncertainty  in  requirements  to  resolve  over  time  and  endeavor 
only  what  is  immediately  achievable.  The  Government  Accountability  Office 
(GAO)  has  also  urged  the  DoD  to  move  toward  Knowledge-Based  Acquisition, 
with  Technology  Readiness  Levels  (TRL)  as  the  rubric  for  program  initiation 
(advanced  development)  (GAO,  1999a).  Thus,  at  the  very  heart  of  EA  is  the 
exclusive  use  of  mature  technology  to  reduce  project  scope. 


Observations  and  Assessments: 

Implications  of  the  Policy 

We  have  managed  and  observed  development  efforts  that  employed 
evolutionary  development  approaches  successfully.  Two  development 
programs  of  the  1990s— the  Army  Tactical  Missile  System  ( ATACMS )  and  the 
Javelin  anti-tank  missile  system— were  compared  herein  with  regard  to  their 
differing  acquisition  strategies,  TRLs,  and  program  outcomes.  Our  study  results 
support  that  a  more  effective  delivery  of  the  first  increment  might  be  facilitated 
with  an  evolutionary  approach.  However,  the  latest  EA  policy  implications 
and  outcomes  are  yet  to  be  fully  known,  and  some  authors  have  already 
expressed  insightful  strategic  and  institutional  oversight  concerns  (Sylvester 
&  Ferrara,  2003).  We  have  also  described  operational  program  management 
concerns  about  its  implementation,  including  excessive  decision  bureaucracy, 
organizational  challenges  from  multiple  and  concurrent  development  efforts, 
outmoded  technology  at  release,  funds  forecasting,  transaction  costs,  and 
maintenance  of  subsequent  increment  priority  (Dillard,  2005).  Our  additional 
findings  suggest  that  incremental  development  may  not  be  appropriate  as  a 
one-size-fits-all  strategy. 


VARIETY  AND  COMPLEXITY 

For  example,  variety  and  complexity  are  elements  of  project  risk.  While 
variety  and  product  diversity  are  preferred  by  market  consumers  to  satisfy 
mainstream  and  smaller  niche  needs,  variety  adds  complexity  in  production 
and  is  costly  for  hardware  manufacturers  and  owners  alike.  In  support  of  EA 
policy,  the  GAO  has  often  used  product  examples  such  as  home  appliances  and 
commercial  vehicles,  which  tend  to  ignore  product  variety  from  the  vantage 
point  of  fleet  owner  vs.  that  of  the  producer  (GAO,  1998a,  1998b,  1999b, 
2003).  The  DoD  is  unique  in  that  it  almost  entirely  outsources  capital  projects 
for  exclusively  internal  use,  and  this  aspect  of  lifelong  ownership  of  an  entire 
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fleet  of  systems  presents  a  different  relationship  than,  for  example,  a  product 
manufacturer  may  have  with  its  production  aircraft. 

Traditional  views  about  variety  from  late  design  changes  are  usually  negative, 
except  for  producibility  savings  and  performance  enhancements.  Changing 
production  configurations  is  not  viewed  as  efficient  due  to  supportability  issues 
(regarding  spares  and  maintenance)  with  lot,  model,  and  type  diversity.  RAND’s 
study  of  support  considerations  for  the  current  mixed  configuration  fleet  of 
Unmanned  Aerial  Vehicles  (UAV)  reported,  “Multiple  aircraft  configurations 
drive  multiple  spare  component  packages  and,  in  the  most  extreme  cases,  may 
drive  multiple  pieces  of  test  equipment,  all  significantly  increasing  long-term 
support  costs”  (Drew,  Shaver,  Lynch,  Amouzegar,  &  Snyder,  2005;  emphasis 
added).  Reliability  issues  can  also  emerge  because  of  insufficient  testing  of 
the  changes.  Depending  on  the  degree  of  change,  system  validation  and 
qualification  become  a  concern  if  changes  are  not  under  strict  control.  There 
may  be  backward  compatibility  and  interoperability  issues  as  well.  Another 
burden  is  the  training  impact  of  mixed  capabilities  within  the  force  or  even 
within  the  same  owning  and  operating  unit. 

Higher  levels  of  risk  from  system  complexity  are  generally  believed  to  be 
mitigated  by  control  measures,  as  within  organizational  contingency  theory  (i.e., 
centralization/decentralization,  etc.).  The  American  nuclear  Navy  was  rooted  in 
CAPT  Hyman  G.  Rickover’s  visit  to  Oak  Ridge  National  Laboratory  in  1946  to 
investigate  the  feasibility  of  using  nuclear  power  aboard  submarines.  During 
his  long  tenure  as  head  of  the  nuclear  program,  he  maintained  fundamental 
principles  about  technical  and  organizational  program  structures,  not  the  least 
of  which  was  personal  accountability.  Those  who  have  worked  with  acquisition 
of  nuclear  plant  materials  know  well  both  the  specifications  and  standards 
of  quality  that  are  unique  to  this  commodity,  as  well  as  Rickover’s  tenets  of 
responsibility  and  accountability  that  are  still  in  place.  They  are  largely  believed 
to  be  important  aspects  of  how  he  successfully  dealt  with  the  complexities 
and  uncertainties  of  a  new  application  of  technology.  The  Guide  to  the  Project 
Management  Body  of  Knowledge  (PMBOK)  (Project  Management  Institute, 
2004)  also  asserts  that  change  in  the  course  of  projects  and  products  is 
inevitable  and  mandates  the  need  for  a  disciplined  change-control  process  to 
control  its  impacts— from  inception  to  completion.  Many  other  useful  theorems 
on  systems  complexity,  change,  and  control  exist  to  alleviate  unwanted  variation 
in  development  and  production. 


CYCLE-TIME  AND  PHASE  CONCURRENCY 

We  have  observed  that,  though  concurrency  is  a  necessary  ingredient  for 
efficient  project  management,  it  has  also  long  been  correlated  with  risk  (due 
to  interdependence  of  activities)  and  might  vary  significantly  with  the  types  of 
activities  underway— inferring  that  periods  of  stable  production  configuration 
between  development  increments  reduce  complexity  in  program  structure  and 
attendant  risks.  Cycle-time  for  the  development  of  each  increment,  and  the 
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relatively  successive  or  concurrent  phasing  of  the  follow-on  increments,  will  also 
have  a  definite  impact  on  program  structure,  budgeting,  project  complexity,  and 
organizational  issues,  etc.  For  reasons  that  we  have  brought  forth  in  our  work 
on  the  computational  modeling  of  evolutionary  development,  we  have  concerns 
about  the  conduct  of  incremental  development  programs  with  continual  and 
highly  concurrent  phasing  of  development  increments. 


The  more  projects  that  specialists  support,  the 

LESS  THEY  ARE  PROPORTIONATELY  AVAILABLE  TO  THE 
PROJECTS  DUE  TO  "QUEUING  INEFFICIENCIES.” 


Particularly  in  matrix  organization  structures,  as  is  often  the  case  with 
projects,  there  can  be  a  tendency  to  staff  multiple  projects  with  a  single 
specialist.  The  more  projects  that  specialists  support,  the  less  they  are 
proportionately  available  to  the  projects  due  to  “queuing  inefficiencies.”  Their 
availability  decreases  because  of  the  need  for  transition  between  projects 
(physical,  mental,  learning  curve,  etc.).  This  has  at  times  resulted  in  large  delays 
in  project  completion  (Smith  &  Reinhartsen,  1998).  Similarly,  Ibrahim  (2005)  has 
shown  that  discontinuous  enterprise  membership  is  a  contributing  factor  toward 
knowledge  loss  in  organizations  involved  in  large  complex  product  development 
processes.  Examining  knowledge  flows  across  product  life  cycles  reveals  that 
members  often  are  not  engaged  in  all  phases.  Whether  from  rotation  of  duties 
or  multi-tasking,  a  discontinuous  member’s  inaccurate  knowledge  could  cause 
a  functional  error  at  the  individual  level,  which  is  not  immediately  obvious  at 
the  enterprise’s  overall  project  level.  Ibrahim’s  findings  support  observations  of 
knowledge  loss  continuing  despite  investments  in  information  technology  and 
knowledge  management. 


Development  Case  Studies 

One  of  the  most  recent  monographs  we  found  on  emerging  results  of 
evolutionary  acquisition  is  by  RAND— on  five  immature,  non-man-rated  space 
systems.  Space  systems  are  somewhat  different  than  general  force  defense 
projects  (in  their  quantities  produced,  their  operational  space  environment, 
greater  proportional  front-end  investment,  and  technology  development 
periods).  RAND  also  found  that  evolutionary  policy  confusion  persists  and  that  EA 
added  program  complexity  and  uncertainty,  especially  with  regard  to  budgeting. 
Extending  their  findings  to  non-space  DoD  programs,  RAND  highlighted  the  EA 
challenges  of  programmatic  flux  (Drew,  Shaver,  Lynch,  Mahyar,  &  Snyder,  2005). 
They  feel,  and  we  agree,  that  EA  presents  the  opportunity  for  typical  non-space 
project  management  challenges  to  be  even  more  formidable. 
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For  such  traditional  defense  systems,  as  expository  cases  of  evolutionary 
acquisition,  we  analyzed  two  tactical  missile  programs  that  illustrate  both 
planned  and  unplanned  change:  the  Army  Tactical  Missile  System  (ATACMS) 
and  the  Javelin  Anti-Tank  Weapon  System  (Dillard  &  Ford,  2007).  Both  of  these 
systems  began  as  Defense  Advanced  Research  Projects  Agency  (DARPA) 
programs  in  the  1980s  and  were  fielded  to  forces  and  employed  in  combat  in 
the  1990/2000S.  See  the  full  report  at  http://www.acquisitionresearch. 
net/_files/FY2007/NPS-AM-07-002.pdf  for  a  detailed  description  of  these 
case  studies  and  our  use  of  them  to  investigate  evolutionary  acquisition  with 
computational  modeling. 

ATACMS  employed  both  incremental  and  spiral  strategies  for  product 
development,  benefiting  from  an  elegant,  modular  independent  architecture. 
This  program  was  able  to  omit  its  technology  development  phase  by  employing 
mature  technologies  for  a  leap-ahead  capability  in  range.  The  basic  system 
arrived  essentially  on  budget  and  schedule,  with  several  successive  variants, 
both  pre-planned  and  unplanned.  Years  later,  one  instance  of  a  minor  production 
change  (uncontrolled  variety)  caused  missile  failure  and  a  costly  refit  of  already- 
produced  missiles. 

In  contrast,  Javelin  used  the  single-step-to-full-capability  approach  for 
product  development.  With  much  greater  modular  interdependence,  the 
program  embarked  upon  advanced  development  with  immature  technologies 
in  several  critical  areas— causing  significant  cost  and  schedule  overruns.  The 
system  has  also  experienced  subsequent  design  changes  and  product  variety, 
but  they  have  consisted  more  as  running  production  changes  than  as  product 
variants  or  blocks. 

A  comparison  of  the  development  and  use  of  technology  in  the  ATACMS  and 
Javelin  projects  clearly  illustrates  the  impacts  of  technology  maturity  on  first 
increment  project  performance.  The  Table  compares  the  technology  maturity 
in  the  ATACMS  and  Javelin  projects  by  identifying  the  Critical  Technologies  for 
seven  subsystem  categories  that  both  products  employed.  For  each  subsystem, 
the  Technology  Readiness  Level  of  the  critical  technology  used  at  the  time  of 
insertion  into  the  development  is  shown.  The  ATACMS  project  used  only  critical 
technologies  with  a  minimum  TRL  of  6  and  an  average  of  8.1.  In  sharp  contrast, 
the  Javelin  project  used  technologies  with  a  maximum  maturity  of  TRL6  and 
an  average  of  TRL5.  The  ATACMS  project  used  significantly  more  mature 
technology  than  the  Javelin  project  and  reaped  the  rewards  of  program  success. 

The  relative  cost  and  schedule  performance  of  the  ATACMS  and  Javelin 
projects  reflects  the  differences  in  the  use  of  technology.  The  ATACMS  project 
had  no  Advanced  Development  Phase  Contract  Cost  Growth  and  only  6  percent 
schedule  growth  in  the  Advanced  Development  Phase.  But  the  Javelin  project 
experienced  over  150  percent  cost  growth  and  50  percent  schedule  growth 
in  Advanced  Development.  The  poorer  project  performance  when  less-than- 
mature  technology  was  used  supports  the  potential  effectiveness  of  EA  in 
managing  technology  risk. 
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TABLE.  COMPARISON  OF  PROGRAMS  USING  DIFFERENT 
DEVELOPMENT  APPROACHES  AND  TECHNOLOGY 
READINESS  LEVELS 

Key  Program  Characteristics  -  First  Increment  of  Capability 


Program 

Aspects 

ATACMS 

(Evolutionary) 

Javelin 

(Single-Step) 

DARPA 

Predecessor 

Assault  Breaker  1977-82 

Tank  Breaker  1981-82 

Ultimate 

Capability 

"Deep  Attack’’ 

“Fire  and  Forget" 

Subsystem 

Critical 

Technology 

TRL 

Critical 

Technology 

TRL 

Munition 

Lance  M74 

Bomblet 

9 

Tandem  Shaped 
Charges 

5 

Propulsion 

Solid  Rocket  Motor 

9 

Two-Staged  Solid 
Rocket  Motor 

5 

Flight  Control 

Fin  Surfaces 

9 

Fin  +  Thrust  Vector 
Control  Vanes 

6 

Guidance  and 
Control 

Inertial 

9 

Tracker  Software 
Algorithm 

4 

Safe/Arm  Fusing 

Mechanical 

7 

Electronic 

4 

Software 

Function  (Target 
Acquisition,  Fire 
Control,  etc.) 

Various 

6 

Various 

6 

Sensor 

N/A 

- 

Focal  Plane  Array 

5 

Cost  of  ~$700M 

Development 

~$700M 

Contract  Type  Fixed  Price 

Cost  Reimbursable 

Tech  Development  Phase 

0  Months 

27  Months 

Advanced  Development  Phase— Planned 

48  Months 

36  Months 

Advanced  Development  Phase— Actual 

51  Months 

54  Months 

Total  Time  in  Development 

51  Months 

81  Months 

Advanced  Development  Phase 

Schedule  Growth 

6% 

50% 

Advanced  Development  Phase 

Contract  Cost  Growth 

0% 

150% 
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Synthesis  of  these  cases  reveals  that  as  an  approach  oriented  primarily  on  the 
reduction  of  product  cycle-time,  evolutionary  development  is  highly  facilitated 
by  the  leveraging  of  mature  technologies.  Also,  system  mutability,  along  with 
other  factors  discussed  in  the  next  section— such  as  time  criticality  (user  risk) 
and  modular  interdependency— can  bolster  incremental  development  suitability. 
For  ATACMS,  an  “open,”  or  at  least  elegant,  architecture  was  fundamental 
for  modular  variety,  and  thorough  design  specification  and  configuration 
management  accountability  proved  essential  for  managing  the  complexity  of 
multiple  product  releases.  In  the  case  of  the  Javelin,  key  capabilities  depended 
upon  immature  technologies  and  at  least  one  binary  requirement,  to  the 
detriment  of  project  cost  and  schedule  outcomes.  In  stark  contrast,  modular 
interdependence  was  manifested  by  an  almost  total  system  redesign  for  lengthy 
and  costly  weight  reductions. 


Do  Product  Attributes  Affect  Evolutionary 
Applicability  and  Outcomes? 

More  questions  about  EA  include  whether  products  with  different  attributes 
(e.g.,  hardware  vs.  software,  buildings  vs.  electronics)  may  lend  themselves  more 
or  less  to  the  use  of  an  incremental  development  approach.  From  the  literature 
and  cases  we  examined,  we  offer  the  following  other  product  attributes  for  PMs’ 
careful  consideration  when  planning  product  capability  increments. 


MUTABILITY 

Perhaps  our  foremost  reservation  is  the  appropriateness  of  the  evolutionary 
development  process  for  all  project  sizes  and  product  commodities  in  toto,  and 
the  application  of  the  spiral  process  to  hardware  products  vs.  Dr.  Barry  Boehm’s 
(1985)  original  and  most  relevant  application  of  this  development  approach 
toward  software.  Boehm  himself  warned  of  “hazardously  distinct”  spiral  model 
imitations,  and  in  his  own  words  described  his  vision  of  the  spiral  process: 

The  spiral  development  model  is  a  risk-driven  process  model  generator. 
It  is  used  to  guide  multi-stakeholder  concurrent  engineering  of 
software-intensive  systems.  It  has  two  main  distinguishing  features. 
One  is  a  cyclic  approach  for  incrementally  growing  a  system’s  degree 
of  definition  and  implementation  while  decreasing  its  degree  of  risk. 
The  other  is  a  set  of  anchor  point  milestones  for  ensuring  stakeholder 
commitment  to  feasible  and  mutually  satisfactory  system  solutions 
(Boehm  &  Hansen,  2000,  p.  3). 

Clearly,  the  conceiver  of  this  spiral  notion  was  oriented  upon  amorphous 
requirements  and  continuous  stakeholder  inputs  for  the  alleviation  of  project 
risk  with  a  very  mutable  product  (Boehm,  1988).  The  nature  of  software  being 
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in  the  digital  rather  than  physical  realm,  it  is  particularly  conducive  to  rapid  and 
successive  revision  and  nearly  costless  production.  And,  Boehm  encourages 
varying  from  the  spiral  model  as  needed  and  reverting  to  a  sequential  model  if 
requirements  are  well  established  and  the  project  less  risky. 

Multiple  product  increments  do  not  often  appear  in  large,  static,  singular 
projects  such  as  bridges,  highways,  office  buildings,  or  in  other  project  areas 
that  have  typically  long  lead  times  or  product  cycles,  such  as  feature-length 
films,  pharmaceuticals,  etc.  These  are  what  we  call  nearly  immutable  products 
and  are  much  different  than  smaller  projects  (like  rapid  software  application 
development)  with  much  shorter  development  periods.  However,  as  with  almost 
everything  engineered  that  we  can  observe  in  the  physical  world,  even  these 
things  can  evolve  and  change  with  additions,  spin-offs,  sequels  (and  prequels), 
expansions,  etc.  Mutability  simplifies  change,  and  that  idea  can  be  extended  to 
many  DoD  projects. 


USER  RISK/SAFETY 

For  DoD,  product  attributes  that  are  aligned  with  Boehm’s  notion  of  process 
models  being  driven  by  risk  are  those  of  mission  or  time  criticality,  survivability, 
and  user  safety.  System  safety  is  often  described  in  terms  of  “man-rating”  as 
approval  for  safe  usage. 

Time-critical  or  enhanced  survivability  systems.  DoD’s  products  have  expanded  risk 
considerations  beyond  Boehm’s  models  of  commercial  software.  Extending  the 
idea  of  project  risk-as-a-driver  down  to  the  level  of  the  end  user,  it  might  seem 
logical  to  assume  that  time  criticality  of  the  need  or  mission  (in  which  risk  of 
not  achieving  project  success  actually  endangers  customer  lives),  might  be  a 
significant  factor  in  the  appropriate  application  of  the  evolutionary  process 
for  reduced  initial  product  cycle-time.  Perhaps  defensive  systems  are  a  good 
example.  The  immediate  need  for  a  Rocket-Propelled  Grenade  defeater  or 
an  Improvised  Explosive  Device  neutralizer  for  currently  deployed  forces  in 
Iraq  and  Afghanistan,  for  example,  clearly  dictates  that  lives  will  be  lost  if  a 
near-term  capability  is  not  achieved.  We  also  cite  as  an  example  the  National 
Missile  Defense  initiative  in  which,  given  the  view  of  near-term  threats,  early 
deployment  of  even  rudimentary  capability  has  been  deemed  preferable 
to  waiting  for  full  capability.  Such  urgency  likely  precludes  full  and  certain 
requirements  specificity. 

Non-man-rated  and  man-rated  systems.  In  the  same  vein,  non-man-rated  systems 
(i.e.,  UAVs  or  cave-exploring  robots— capabilities  in  which  operator  lives  are  not 
at  risk  if  the  product  fails)— may  also  lend  themselves  readily  to  rapid  innovation 
and  riskless  experimentation  cycles.  However,  user  hazard  levels  for  man-rated 
systems  may  be  an  entirely  different  matter. 
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Man-rated  systems  present  a  different  challenge.  Configuration  variety 
adds  technical  complexity  with  sometimes  unpredictable  interactions.  In  such 
projects  as  pharmaceuticals,  aviation,  vehicular  transportation,  etc.,  producers 
mitigate  safety  risks  with  thorough  analyses,  documentation  reviews,  validation 
testing,  and  other  control  and  verification  processes.  By  their  very  nature— with 
lethal  hazards  for  the  end  user  and  typically  lengthy  approval  criteria— these 
may  not  be  good  candidates  for  an  evolutionary  approach. 


LOGISTICAL  SUPPORT  PLANNED  DURING  SERVICE/SHELF  LIFE 

Our  observations  warn  that  multiple  configurations  of  hardware  products 
come  at  a  cost  for  fleet  ownership.  Veterans  of  new  system  deployments  across 
the  force/fleet,  or  throughout  any  large  using  organization,  know  the  difficulties 
of  rolling  out  a  configuration  change.  Benefits  of  standardization  have  long 
been  offered  via  production  economies  of  scale,  commonality  of  parts  across 
platforms,  and  interoperability.  If  the  ultimate  goal  is  to  have  standardization 
across  the  DoD’s  force,  owning  multiple  configurations  (variety)  of  a  system 
seems  in  opposition,  with  added  complexity  in  training  and  supply  support  of 
the  item.  The  logistical  maintenance  strategy  cannot  be  ignored— whether  the 
end-item  is  maintenance-intensive,  such  as  tactical  vehicles,  or  maintenance- 
free,  such  as  with  many  electronics  items  and  munitions,  and  situations  in  which 
physical  changes  are  completely  transparent  to  the  user.  For  multiple-product 
configurations,  the  acquisition  approach  could  have  a  huge  effect  on  the  total 
costs  of  ownership,  as  previously  mentioned  by  Drew  et  al.  (2005)  in  regard  to 
UAVs. 


RANGE  OF  REQUIREMENT  ATTAINMENT 

Most  requirements  are  “continuous,”  i.e.,  may  be  satisfied  in  varying  amounts 
of  attainment.  Thus,  ranges  of  their  satisfaction  can  be  flexibly  specified,  allowing 
for  thresholds  (minimum  values  of  attainment)  and  objectives  (optimal  values 
of  attainment).  Examples  are  range,  accuracy,  weight,  reliability,  etc.  However, 
we  have  found  that  some  requirements,  often  critical  ones,  are  more  binary  in 
nature  than  continuous.  They  have  a  much  narrower  range  of  attainment,  such 
that  they  are  essentially  pass/fail  or  go/no-go  in  their  demonstration.  Examples 
are  Windows-compatibility,  “soft”  missile  launch,  network  security,  physical  fit, 
leak-proof,  shock-/vibration-/drop-proof,  survivability,  horizontal-to-vertical 
flight  transition,  etc.  If  one  of  these  more  binary-type  requirements  happens  to 
be  a  Key  Performance  Parameter,  its  attainment  will  be  on  the  project’s  critical 
path  and  highly  dependent  upon  technical  maturity.  As  such,  it  might  practically 
dictate  the  length  of  the  entire  advanced  development  effort  and  make  division 
into  capability  increments  less  beneficial  as  a  development  strategy.  Though 
somewhat  correlated  with  product  reliability,  these  kinds  of  requirements 
demand  a  system  that  “either  works  or  it  doesn’t”  without  the  flexibility  afforded 
by  objectives  and  thresholds. 
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AMOUNT  OF  CHANGE- AND  THE  LURE  OF  MODULARITY 

We  subscribe  to  the  current  systems  theorists’  view  that  complexity  is 
comprised  of  numbers  (of  components),  connections  (interdependencies) 
and  distinctions  (variety).  Distinction  corresponds  to  variety,  to  heterogeneity, 
and  to  the  fact  that  different  parts  of  complex  systems  behave  differently 
(Heylighen,  1997).  Variety  is  a  component  of  Nobel  Prize  winner  Herbert  Simon’s 
explanation  of  complexity— many  different  parts  with  many  interactions.  Simon 
argued,  from  his  observation  of  complexity  in  things  both  natural  and  artificial, 
that  complex  systems  evolve  from  simple  systems.  And,  they  do  so  more  rapidly 
when  there  are  stable,  intermediate  forms  or  sub-systems  (like  modules  or  “units 
of  action”)  (Simon,  1981).  Moreover,  he  argues  the  resulting  evolution  into  the 
complex  system  will  be  hierarchical.  Earlier,  in  The  Architecture  of  Complexity, 
Simon  (1962)  proposed  hierarchy  as  a  universal  principle  of  complex  structures. 
He  felt  that  complex  problems  could  be  solved  more  easily  when  decomposed 
into  sub-problems  (similar  to  how  project  managers  employ  Work  Breakdown 
Structures  via  the  Systems  Engineering  Process).  And,  sub-problem  solutions 
could  be  combined  into  a  solution  for  larger  problems,  etc. 

Commonly  seen  today  are  modular  industrial  products  that  are  sometimes 
designed  as  complete  architectures,  with  standardized  interfaces  that  invite 
others  to  introduce  complementary  products  for  insertion  (Agre,  2003).  The 
Modular  Open  Systems  Approach  (MOSA)  is  a  relatively  new  DoD  initiative  that 
encourages  the  use  of  widely  supported  commercial  interface  standards  and 
disciplined  interface  controls  to  develop  systems  architectures  using  modular 
design  concepts  (DoD  Open  Systems  Joint  Task  Force,  2004). 

As  in  biological  evolution,  improved  “fitness”  with  a  system’s  environment 
is  sought  in  the  adapting  or  evolving  of  systems.  But  others  have  noted  that 
Simon’s  metaphors  for  dynamic  complex  systems,  useful  as  they  are  for 
understanding  complexity,  fall  short  of  explaining  their  evolution.  While  the 
concept  of  modularity  suggests  approximately  independent  subsystems  may 
be  modified  or  adapted  as  such,  it  has  been  shown  that,  in  the  aggregate,  there 
is  yet  quantifiable  modular  interdependency  that  affects  evolvability  (Watson  & 
Pollack,  2005).  In  other  words,  how  changes  in  the  state  of  one  module  affect 
the  state  of  another  is  relative  and  measurable.  Thus,  Simon’s  writings  illustrate 
that  modularity  is  beneficial  for  production  but  not  necessarily  for  evolution. 

Examples  of  modular  interdependency  are  plentiful.  In  the  aircraft  or 
automotive  realm,  an  engine  upgrade  would  intuitively  seem  to  be  a  relatively 
independent  subsystem  change.  But,  systems  engineers  know  that  changes 
propagate  through  hardware  almost  as  much  as  software  in  the  long  run— just 
as  the  eventual  rise  in  building  temperature  from  the  thermostat  adjustment 
in  one  modular  room.  For  instance,  adding  increased  armor  protection  (and 
weight)  for  deployed  High  Mobility  Multi-purpose  Wheeled  Vehicles  has  resulted 
in  increased  wear-out  of  drive  train  and  suspension  components  and  impacts  to 
vehicle  range,  mobility,  mileage,  etc.  As  a  result,  “up  armor”  kits  have  become 
only  a  stopgap  measure  until  totally  redesigned  systems  can  be  produced. 
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Thus,  we  suggest  it  is  not  only  the  structural  modularity  and  standard 
interfaces  that  enable  system  evolution;  but,  it  is  also  the  relative  interdependency 
of  the  modules.  In  short,  PMs  need  to  be  mindful  of  the  degree  of  change  in 
subsequent  increments/spirals.  One  subsystem  is  likely  to  affect  another  in 
the  short-  or  long-run.  And,  that  can  make  product  evolution  problematic.  As 
Norman  Augustine  once  said,  “No  change  is  a  small  change”;  to  convey  that 
independent  subsystems,  even  redundant  ones,  aren’t  always  independent 
(Augustine,  1997). 


PRODUCTION  QUANTITY 

Many  might  correlate  the  applicability  of  evolutionary  development  with  long 
production  runs.  But  we  have  also  collaborated  with  officials  from  NASA  who 
have  said,  “No  two  identical  spacecraft  are  the  same,”  which  seems  to  contradict 
any  idea  that  like  configurations  are  a  necessity  among  small  production  lot  sizes 
(Roy,  2006).  Indeed,  naval  shipbuilders  voice  the  same  about  variation  among 
individual  ships,  or  within  flights,  of  the  same  class.  And  even  one-of-a-kind, 
nearly  immutable  projects  like  skyscrapers  and  bridges  can  be  later  remodeled 
and  refitted,  as  discussed  earlier.  Aside  from  truly  singular  efforts,  we  have  not 
yet  found  any  universal  evidence  of  an  evolutionary  approach  being  more  or 
less  suitable  according  to  quantity  of  systems  produced. 


Recommendations  for  Practice 

Project  managers  need  to  be  aware  of  the  inherent  risks  of  evolutionary 
development  and  take  necessary  precautions  to  balance  those  risks.  Many  tools 
and  control  measures  are  developed  and  available  to  assist  project  managers 
in  balancing  the  risks,  such  as  TRLs,  technical  performance  measurement, 
technical  reviews,  modeling  and  simulation,  real  options,  project  phasing,  risk 
management,  configuration  management,  earned  value  management,  and 
organizational  design. 

Incremental  development  projects  require  steps  to  alleviate  risks  that  may 
be  inherent  in  the  program  structure.  These  include  decisions  about  the  number 
and  concurrency  of  development  increments  and  their  scope  and  impact  on  the 
organization  staffing. 

Product  attributes  may  help  determine  the  suitability  of  evolutionary 
development.  PMs  should  consider  characteristics  such  as:  mutability,  time 
criticality,  man-rating,  modular  interdependency,  key  parameters  of  capability 
vs.  range  of  requirement  attainment  (i.e.,  binary  vs.  continuous),  and  the  relative 
amounts  of  modular  interdependency  in  the  system  architecture. 

Rigorous  configuration  management  accountability  must  be  assigned 
and  maintained  for  supportability,  reliability,  failure  mode  identification  and 
causality,  and  to  prevent  the  variety  generated  by  EA  from  reducing  total 
product  performance. 
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Conclusions 

Dr.  Barry  Boehm’s  recent  book  (2004)  on  software  development  advocates 
balancing  disciplined  (more  rigid)  and  agile  (more  flexible)  methods  to  capitalize 
on  the  benefits  of  both.  Discipline  is  needed  as  a  control  mechanism  to  avoid 
risk,  but  agility  is  needed  to  respond  quickly  to  customer  needs.  Saying,  “One 
size  fits  all  is  a  myth,”  he  advocates  a  balanced  approach  based  upon  risk. 
Consistent  with  our  findings,  he  also  advocates  more  disciplined,  risk-averse 
approaches  for  projects  that  are  mission/safety  critical,  larger  in  size,  and  have 
more  stable  requirements. 

It  could  be  summarized  that  evolutionary  development  was  at  its  inception, 
and  is  at  its  extension,  all  about  risk.  Paradoxically,  it  is  an  agile  method 
envisioned  to  reduce  risk  and  yet  can  potentially  add  its  own.  On  the  one  hand, 
a  spiral  or  incremental  approach  allays  risk  by  reducing  scope  to  render  only 
the  highest  priority  capabilities  with  the  exclusive  use  of  mature  technology; 
and  obtains  early  and  continuous  feedback  from  the  environment  for  follow- 
on  developments.  On  the  other  hand,  it  introduces  concurrency  during 
advanced  development  and  adds  variety  in  production,  with  all  their  attendant 
management  challenges. 

We  have  suggested  that  a  one-size-fits-all  methodology  for  DoD  system 
development  may  not  be  appropriate,  and  we  have  offered  for  consideration 
several  product  attributes  that  might  help  determine  the  applicability  of  agile 
approaches.  We  speculate  that  evolutionary  development  may  serve  better  than 
single-step  development  for  initial  capability  when  products  are  mutable,  time- 
critical,  non-maintenance-intensive,  and  have  continuous  (vs.  binary)  or  uncertain 
requirements,  short  cycle-times  (less  knock-on  effects),  sequentially  phased 
development  blocks,  and  modular  independence.  In  contrast,  evolutionary 
development  may  not  be  as  suitable  when  there  are  product  safety  or  man¬ 
rating  concerns  and  attributes  opposite  to  those  discussed  here.  In  particular, 
PMs  should  understand  the  nature  of  their  product  requirements  with  regard 
to  their  range  of  attainment  and  relative  to  key  parameters  of  capability  and 
vis-a-vis  the  readiness  level  of  their  enabling  technologies.  Some  key  features 
may  indeed  be  binary,  and  others  may  have  significant  ramifications  of  partial 
attainment— such  as  propagated  change  across  the  entire  product  componentry 
(as  in  weight  reduction)  vs.  a  more  independent  modular  modification. 

Variety  can  be  both  an  asset  (for  end-users)  and  a  liability  (for  manufacturers, 
owners,  and  supporters).  As  such,  to  compensate  for  product  variety  risk, 
we  posit  that  acquirers  must  “own”  the  design  and  emphasize  configuration 
management,  keeping  or  assigning  responsibility  for  that  function  and 
maintaining  accountability  for  it  (i.e.,  explanation  of  how  assigned  functions  are 
being  met). 

Our  title— “from  amorphous  to  defined”— alludes  not  only  to  product 
specification,  but  also  to  risk  realization  in  evolutionary  development.  PMs 
must  be  aware  it  has  inherent  challenges,  both  strategic  and  tactical;  they 


2  5  0 


A  Publication  of  the  Defense  Acquisition  University 


www.dau.mil 


must  balance  them  with  tools  that  we  have  mentioned.  In  this  article,  we  have 
both  highlighted  and  illustrated  them,  as  well  as  showing  that  incremental  and 
spiral  development  can  indeed  work  well— especially  for  technically  mature  and 
mutable  products  with  open  or  elegant  architecture. 

Finally,  stability  is  the  quest  in  all  things  programmatic:  for  funding, 
requirements,  design,  production  configuration,  etc.  But  in  an  unstable  world, 
and  with  the  future  filled  with  uncertainty,  the  only  constant  is  likely  to  be 
change,  and  tension  between  control  and  change  is  probably  unending.  PMs 
do  have  some  tools  for  coping,  and  being  forewarned  is  forearmed.  Successful 
use  of  these  tools  to  balance  control  and  risk  in  projects  with  a  high  rate  of 
change  and  concurrency  is  an  area  for  further  research,  to  improve  both  our 
understanding  and  use  of  evolutionary  acquisition. 
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